Active Set Algorithm for Structured Sparsity-Inducing Norms

We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual l1-norm and the group l1-norm by allowing the subsets to overlap. This leads to a specific set of allowed nonzero patterns for the solutions of such problems. We first explore the relationship between the groups defining the norm and the resulting nonzero patterns. In particular, we show how geometrical information about the variables can be encoded by our regularization. We finally present an active set algorithm to efficiently solve the corresponding minimization problem.